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Report  Lo  the  Chairman,  Committee  on 
kScience,  Space,  and  Technology,  House  of 
t^epresentatiyes 
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SYSTEM 

Broader  Involvement  of 
the  EOSDIS  User 
Community  Is  Needed 
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United  States 

General  Accounting  Offlce 

Washington,  D.C.  20548 


Information  Management  and 
Technology  Division 

B-247881 

May  11, 1992 

The  Honorable  George  E.  Brown,  Jr. 

Chairman,  Committee  on  Science,  Space, 
and  Technology 
House  of  Representatives 

Dear  Mr.  Chairman: 

You  requested  that  we  review  various  aspects  of  the  National  Aeronautics 
and  Space  Administration’s  (NASA)  Earth  Observing  System  Data  and 
Information  System  (eosdis).  NASA  has  initiated  a  program,  called  the 
Earth  Observing  System  (EOS),  to  collect  an  integrated  set  of  data  to  study 
the  earth’s  atmosphere,  biosphere,  oceans,  land  surfaces,  and  polar 
regions  as  a  complete  system  over  a  15-year  period.  The  data  from  EOS  will 
be  processed,  archived,  and  distributed  by  EOSDis.  As  part  of  yoim  request, 
you  asked  us  to  determine  the  extent  to  which  users  have  provided  input  in 
defming  EOSDIS  requirements.  This  report  describes  the  degree  of  user 
involvement  and  its  importance  in  the  EOSDIS  planning  and  development 
process.  Details  of  our  objectives,  scope,  and  methodology  are  provided  in 
appendix  I. 


Results  in  Brief 


NASA  has  not  done  enough  to  ensure  that  input  from  the  full  range  of 
anticipated  EOSDIS  users  is  incorporated  in  the  system’s  design.  The 
predominant  users  of  eosdis  are  expected  to  be  the  thousands  of  earth 
scientists  who  are  not  affiliated  with  the  EOS  program.  However,  NASA’s 
planning  for  the  system  thus  far  has  relied  largely  on  input  from  the 
relatively  small  number  of  researchers  funded  directly  by  NASA.  NASA’s 
guidelines  and  mechanisms  for  obtaining  further  user  input  in  the  future  do 
not  provide  assurance  that  all  segments  of  the  user  community  will  be 
adequately  represented.  Because  of  the  uncertainty  regarding  the  total 
number  of  ultimate  eosdis  users  and  the  system  work  loads  they  will 
generate,  a  system  development  strategy  is  needed  that  can  adapt  to 
changing  user  needs. 


Background 


In  recent  years,  complex  scientific  questions  and  policy  issues  have 
emerged  concerning  the  effect  of  natural  earth  proce.sses  and  human 
activities  on  the  environment.  These  issues  and  the  need  for  accurate  and 
comprehensive  scientific  information  on  which  to  base  environmental 
policy  decisions  have  led  to  the  creation  of  a  number  of  research  initiatives, 
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including  the  U.S.  Global  Change  Research  Program.  There  are  nine 
federal  agencies,  including  NASA,  in  this  program.' 

EOS,  a  series  of  space-based  observatories  containing  a  variety  of 
instruments  that  will  collect  data  about  the  earth,  is  NASA’s  main 
contribution  to  the  Global  Change  Research  Program.  A  key  component  of 
the  program  is  the  $3  billion  EOSDIS.  It  will  be  the  system  responsible  for 
archiving  and  distributing  all  NASA  earth  science  data,  both  from  past  and 
current  missions  and,  in  the  future,  from  the  EOS  satellites.  The  information 
from  EOSDIS  is  intended  to  enable  the  scientific  community  and  researchers 
worldwide  to  broaden  their  understanding  of  environmental  change  within 
the  entire  earth  system  and  to  improve  their  ability  to  predict  future 
change. 

In  the  past,  space  research  emphasized  short-lived,  single-purpose 
missions  designed  to  provide  a  first  generation  of  data  to  a  few  scientific 
specialists.  The  objectives  of  each  mission  were  usually  independent  of  the 
objectives  of  other  missions.  Planned  future  missions,  such  as  EOS,  have 
adopted  a  more  complex  multidisciplinary  approach  in  order  to  gather  data 
to  address  multifaceted  objectives  and  answer  broad  scientific  questions. 
Emphasis  is  generally  shifting  from  single-investigator  exploration  to 
collaborative  efforts  of  various  teeuns  of  investigators  and  other  global 
change  researchers. 

NASA  released  a  request  for  proposals  (RFP)  for  development  of  mjgor 
components  of  EOSDIS  in  July  1991 ,  after  7  years  of  conceptual  design  and 
requirements  studies.  The  agency  intends  to  award  a  contract  for  detailed 
design,  development,  and  installation  of  the  system  in  November  1992.  An 
important  aspect  of  the  design  and  eventual  success  of  EOSDIS  will  be  the 
degree  of  flexibility  with  which  the  system  can  accommodate  changing 
user  needs  and  advances  in  technology.  NASA  plans  an  evolutionary 
approach  to  EOSDis  development,  including  prototyping  and  a  series  of 
system  versions,  each  adding  incrementally  to  the  previous  versions. 

NASA’s  plans  for  prototyping  and  use  of  advanced  technology  are  discussed 
in  a  previous  GAO  report.^' 


'The  other  eight  federal  agencies  are  the  National  Science  Foundation,  Natioiud  Oceanic  and 
Atmospheric  Adirunistration,  U.S.  Geological  Survey,  Department  of  Agriculture,  Environmental 
Protection  Agency,  Department  of  Energy,  Smithsonian  Institution,  and  Department  of  Defense. 

^Earth  Observing  System:  NASA's  EOSDIS  Development  Approach  Is  Risky  (GAO/lMTEC-92-24,  Feb. 
2.5,  1992). 
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The  anticipated  users  of  EOSDIS  include  a  broad  range  of  scientists  and 
other  users  that  we  categorized  into  three  groups:  NASA-affiliated, 
nonaffiliated,  and  “other.”  The  NASA-affiliated  group  consists  of  the  551 
EOS  investigators,  EOS  operations  personnel  (e.g.,  EOS  program  and  project 
officials,  data  center  managers,  and  project  scientists),  and  other 
NASA-funded  researchers.  The  nonaffiliated  group  is  comprised  of 
researchers,  not  funded  by  NASA  or  the  EOS  program,  at  universities  and 
U.S.  and  foreign  government  agencies.  The  “other”  user  category  includes 
policymakers  at  the  federal,  state,  and  local  levels;  commercial 
researchers;  and  users  applying  earth  science  data  for  public  policy 
purposes.^ 


Largest  User  Group 
Has  Had  Minimal 
Involvement  in 
Development  of 
EOSDIS  Requirement 
and  Planning 
Documents 


NASA  expects  the  largest  number  of  EOSDIS  users  to  be  nonaffiliated 
scientists.  However,  this  group  has  had  minimal  involvement  in  defining 
EOSDIS  requirement  and  planning  documents.  Despite  recommendations 
from  scientific  committees  to  obtain  a  broad  spectrum  of  user  involvement, 
NASA  primarily  obtained  input  from  EOS  investigators  and  other 
NASA-affiliated  users. 


Science  Community’s 
Guidance  Stresses  Broad 
User  Involvement 

DTIC  QUALIIT  i;  5 


Acoesslon  For 


The  science  community,  including  the  National  Academy  of  Sciences  and 
the  EOS  Science  Steering  Committee,  recommends  broad  user  involvement 
throughout  the  development  of  earth  science  data  systems.  Several  reports 
emphasize  active  involvement  of  scientists  from  inception  to  completion  of 
space  missions.  In  the  past,  there  has  often  been  a  lack  of  scientific 
involvement  in  data  system  design  during  the  early  mission  planning  and 
system  development  phases.  Typically,  the  interdisciplinary  nature  of  data 
has  not  been  fully  recognized,  and  data  systems  frequently  have  not  been 
properly  implemented.  In  several  reports,  the  National  Academy  of 
Sciences  has  stated  that  full  involvement  of  the  broad  user  community  will 


i' 


i 


^NASA  also  categorizes  EOSDIS  users  as  EOS  investigators,  nonaffiliated  users,  auid  “other"  users.  The 
difference  between  our  categories  and  NASA's  categories  is  that  NASA  included  NASA-funded 
researchers  in  its  nonaffiliated  user  group  and  EOS  operations  personnel  in  its  “other"  user  group, 
while  we  included  these  people  in  a  NASA-affiliated  category. 
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maximize  the  science  return  on  space  missions  and  improve  the  quality  of 
data  for  users.^ 

More  specifically,  a  1990  Academy  report  states  that  (1)  excluding  any  of 
the  EOS  user  groups  from  testing  and  decision  making  during  EOSDIS  design 
and  development  would  be  a  m^or  mistake,  and  (2)  when  modifications  to 
original  specifications  are  made,  the  global  research  community  should 
have  a  msyor  role  in  advising  on  priorities.®  The  Academy  considers  broad 
user  input  to  be  essential  to  ensure  that  EOSDIS  priorities  are  based  on 
research  requirements.  The  Academy’s  findings  are  supported  by  a  1987 
report  by  the  EOS  Science  Steering  Conunittee,  which  states  that  eosdis 
needs  to  serve  a  community  that  extends  beyond  those  involved  in  the 
mission.® 

In  addition  to  these  reports,  a  1989  NASA  study,  which  assessed  the 
research  operations  and  information  systems  needs  of  the  space  science 
user  community,  found  that  users  need  to  be  involved  as  the  project 
evolves.^  According  to  the  study,  users  need  to  not  only  be  involved  in 
listing  requirements  but  in  prioritizing  the  requirements  and  in  assessing 
trade-offs. 

Finally,  some  of  the  earth  science  researchers  we  contacted  stated  that  it  is 
important  to  obtain  input  from  users  outside  a  project’s  investigator 
community  because  such  users  can  be  more  objective  than  those 
depending  upon  the  project  for  funding.  Some  of  these  and  other 
researchers  believe  that  if  input  is  not  obtained  from  the  broader  user 
community,  NASA  will  design  a  system  that  may  be  inappropriate  and  too 
inflexible  to  meet  the  needs  of  most  earth  science  researchers. 


^Data  Management  and  Computation,  Volume  1:  lasues  and  ReconunendaUons,  National  Academy  of 
Sciences'  National  Research  Council,  National  Academy  Press,  1982;  lasues  and  Recommendations 
Associated  with  Distributed  Computation  and  Data  Managennent  Systems  for  the  Space  Sciences, 
National  Academy  of  Sciences'  Nkional  Research  Council,  National  Academy  Pre^.  1986;  Selected 
Issues  in  Space  Science  Data  Management  and  Computation,  National  Academy  of  Sciences'  National 
Research  Council,  National  Acaderny  Press,  1988. 

®U.S.  Global  Change  Research  Program:  An  Assessment  of  FY9 1  Plans,  National  Academy  of  Sciences' 
National  Research  Council,  National  Academy  Press,  1990. 

®FYom  Pattern  to  Process:  TTie  Strategy  of  the  Earth  ObeervlnK  System,  EOS  Science  Steering 
Committee  Report,  Volume  11,  National  Aeronautics  and  Space  Administration,  1987. 

^Elaine  R.  Hansen,  George  H.  Ludwig,  Alain  J.  Jouchoux,  Randal  L.  Davis,  University  of  Colorado, 
‘Overarching  Science  Needs  and  Issues:  Shaping  NASA's  Research  Operations  and  Information 
Systems,"  AlAA/NASA  Second  International  Symposium  on  Space  Information  Systems,  (Pasadena, 
California:  Sept.  1 7  - 19,  1990),  American  Inkitute  of  AeronMitics  and  Astronautics,  Washington,  D.C., 
Feb.  1991. 
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Jser  Input  Focused  Almost 
Entirely  on  EOS  Investigators 
nd  Other  NASA-Affiliated 
Jsers 


Although  the  scientific  community  has  recommended  input  from  all  EOSDIS 
user  groups,  NASA  has  focused  almost  entirely  on  EOS  investigators  and 
other  NASA-affiliated  scientists  in  defining  EOSDis  requirement  and  planning 
documents.  NASA  believes  that  the  needs  of  all  EOSDiS  users  have  b^n 
reflected  in  the  development  of  these  dociunents.  However,  we  found  that 
users  not  affiliated  with  NASA  had  minimal  involvement  in  developing  these 
requirement  and  planning  documents. 


During  the  EOSDIS  design  and  development  process,  NASA  developed  three 
main  sets  of  EOSDIS  requirement  and  planning  documents  and  used  various 
mechanisms  to  obtain  user  input.  The  three  main  EOSDIS  requirement  and 
planning  documents  were 


•  the  information  system’s  functional  and  performance  requirements,  which 
establish  the  system’s  architectural,  operational,  performance,  and 
functional  specifications  and  are  part  of  the  RFP; 

•  the  EOS  instrument  data  requirements,  including  descriptions  of  the  data  to 
be  provided  by  EOS  and  the  processing  requirements  needed  to  support 
generating,  archiving,  and  distributing  EOS  data;  and 

•  the  EOSDIS  data  plan,  which  compiles  information  on  data  that  should  be 
accessible  through  BOSDIS  including  global  change  data  at  the  seven 
Distributed  Active  Archive  Centers  (DAACs)  and  earth  science  data  to  be 
generated  before  EOS  is  launched.* 


NASA  used  several  mechanisms  for  determining  and  validating  user  needs. 
These  mechanisms  included  soliciting  advice  from  the  EOSDIS  Advisory 
Panel,  conducting  working  group  retreats,  surveying  EOS  investigators, 
soliciting  information  on  the  draft  functional  requirements,  and  awarding 
two  contractor  studies  on  system  design  requirements. 

NASA  relied  upon  the  EOSDIS  Advisory  Panel  as  a  chief  mechanism  for 
obtaining  user  input.  This  panel  was  established  in  1989;  its  functions 
include  reviewing  eosdis  on  behalf  of  both  EOS  investigators  and  the  broad 
community  of  scientists  expected  to  use  EOS  data.  NASA  believes  that  this 
panel  reflects  the  needs  of  the  earth  science  community  because  its 
members  are  drawn  from  the  551  EOS  investigators,  who  represent  a  wide 
range  of  global  change  research  disciplines.  However,  the  panel’s 


*The  seven  DAACs  will  be  responsible  for  archiving,  processing,  and  distributing  EOS  and  other  earth 
science  data  through  EOSDIS. 
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membership  appears  skewed— only  2  of  its  24  members  are  from  the 
nonaffihated  user  community  and  only  1  is  from  the  “other”  user  group. 


In  addition,  nonaffiliated  and  “other”  users  had  minimal  involvement  in  the 
other  methods  NASA  used  to  develop  the  key  eosdis  requirement  and 
planning  documents,  including  retreats,  a  survey,  solicitations,  and 
contractor  studies.  As  shown  in  figure  1 ,  NASA-affiliated  users  had  primaiy 
involvement  in  determining  user  needs  and  validating  eosdis  requirement 
and  planning  documents,  while  the  nonadfiliated  and  “other”  users  had 
minimal  involvement. 


Figure  1 :  User  Involvement  During  the  Development  of  EOSDIS  Requirement  and  Planning  Documents 
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Note  User  Involvement  with  the  EOSDIS  requirennent  and  planning  documents  was  analyzed  through 
March  1992 


In  addition,  the  nonaffiliated  and  “other”  users  had  minimal  involvement  in 
the  EOSDIS  functional  and  performance  requirement  studies  prepared  by 
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NASA’s  two  conceptual  design  phase  contractors.  As  part  of  this  early 
requirements  definition  process,  NASA  awarded  contracts  to  two  companies 
to  conduct  independent  studies  on  potential  system  designs.  Each 
company  obtained  input  from  the  user  community.  However,  as  shown  in 
figure  2,  of  the  455  users  contacted  by  the  two  contractors,  82  percent 
were  NASA-affiliated  users,  1 7  percent  were  nonaffiliated  users,  and  1 
percent  were  “other”  users. 


gure  2:  Users  Involved  In  System 
iquirement  Definition  Studies 
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Note:  455  users  wore  contacted 

We  contacted  representatives  of  the  nonaffiliated  and  “other”  user  groups, 
including  the  other  federal  agencies  involved  in  the  U.S.  Global  Change 
Research  Program,  the  six  key  earth  science  professional  associations,  a 
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mayor  trade  association  of  industrial  users,  and  other  earth  science 
researchers.®  As  part  of  the  U.S.  Global  Change  Research  Program,  an 
advisory  group  was  established  to  examine  data  management  issues.  None 
of  the  federal  agency  representatives  participating  in  this  advisory  group 
reported  providing  input  in  defining  or  validating  EOSDIS  requirements. 
Furthermore,  none  of  the  seven  associations  we  contacted  said  that  they 
were  asked  to  define  or  validate  EOSDIS  requirements,  although  two  said 
that  this  is  not  an  activity  that  they  would  typically  take  part  in.  Also,  only 
three  of  these  associations  reported  that  NASA  representatives  discussed 
global  change  issues  during  their  aimual  meetings. 

While  several  of  those  contacted  expressed  a  belief  that  NASA  should  be 
able  to  meet  the  needs  of  the  broad  EOSDIS  user  community  based  on  input 
from  the  EOS  investigators,  others  stated  that  it  is  also  important  to  obtain 
input  from  users  outside  the  NASA  investigator  community.  Some  of  these 
association  representatives  and  researchers  believe  that  NASA  may  not  meet 
the  needs  of  the  nonaffiliated  and  “other”  users  by  simply  satisfying  the 
requirements  of  the  EOS  investigators,  'fhey  disagree  with  NASA’s  belief  that 
because  the  EOS  investigators  were  competitively  selected  from  the  earth 
science  research  community  they  represent  the  broad  EOSDIS  user 
community  and  are  knowledgeable  of  their  requirements.  For  example,  one 
researcher  explained  that  the  vast  majority  of  the  nonaffiliated  scientists 
use  less  sophisticated  computer  and  information  systems  than  do  EOS 
investigators  and,  consequently,  their  data  access  requirements  may  vary. 

In  addition,  researchers  have  identified  a  number  of  options  for  involving 
users  outside  the  NASA  research  community.  These  options  include 
(1)  establishing  outreach  programs  at  the  mqjor  universities  with  earth 
science  programs;  (2)  conducting  symposiums;  (3)  addressing  aimual 
conferences  of  the  earth  science  professional  associations,  including 
international  associations;  and  (4)  requesting  that  a  task  force  be 
established  to  review  EOSDIS  requirements. 


®The  earth  science  professional  associations  contacted  were  the  American  Institute  for  Biological 
Science,  American  Geophysical  Union,  American  Meteorological  Society,  American  Association  of 
Ground  Water  Scientists  and  Engineers,  Geological  Society  of  America,  and  the  International 
Association  for  the  Physical  Sciences  of  the  Ocean.  The  trade  association,  representing  industrial  users 
of  remote  sensing  data,  was  the  GEOSAT  Committee, 
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Wider  Science  Conununity  Although  nasa  has  issued  guidance  on  obtaining  future  user  input  during 
Input  Is  Needed  in  Future  development  of  eosdis,  broad  and  comprehensive  involvement  of 

Plans  scientists  outside  the  NASA-affiliated  community  is  needed.  NASA  intends  to 

rely  on  input  from  user  working  groups  located  at  the  seven  daacs.  These 
working  groups  include  some  members  from  the  nonaffiliated  and  “other” 
user  groups.  Beyond  these  groups,  NASA’s  directions  for  future  user  input 
are  contained  in  its  configuration  management  plan  and  the  RFP.  Neither 
the  configuration  management  plan  nor  the  RFP  adequately  takes  into 
account  all  segments  of  the  user  community. 

NASA’s  EOS  Project  Configuration  Procedures  Handbook  addresses  how 
changes  are  to  be  made  to  hardware  design,  software,  and  other  system 
specifications.  The  handbook  identifies  the  roles  of  NASA  headquarters,  the 
EOS  project  office,  EOS  investigators,  and  contractors.  However,  it  fails  to 
address  the  role  or  the  mechanism  for  input  by  users  outside  the  NASA 
community. 

Likewise,  the  RFP  does  not  specificaUy  direct  the  contractor  to  obtain  input 
from  the  nonaffiliated  and  “other”  users.  However,  the  RFP  does  ask  the 
contractor  to  describe  how  the  system  will  be  designed  to  facilitate  use  by 
the  nonaffiliated  and  “other”  users.  Some  earth  science  researchers  believe 
the  RFP  language  is  too  broad  and  vague  in  discussing  how  the  contractor 
will  work  with  users.  For  example,  one  researcher  believes  the  contractor 
will  expend  minimal  effort  to  obtain  user  input  from  nonaffiliated  and 
“other”  user  groups  unless  specificaUy  required  to  do  so  by  the  contract. 


User  and  Work  Load 
Estimates  Are 
Uncertain 


The  expected  number  of  eosdis  users  and  the  work  load  these  users  will 
generate  are  not  certain.  NASA  issued  preliminary  user  and  work  load 
estimates  after  completing  m^or  portions  of  the  conceptual  development 
and  system  requirements  process.  The  estimates  are  used,  in  part,  to 
provide  the  contractor  with  a  basis  for  designing  and  developing  eosdis. 

NASA  prepared  two  preliminary  eosdis  user  estimates  that  varied  widely 
and  have  not  been  incorporated  into  an  official  EOSDIS  requirement 
document.  A  1990  user  estimate  contained  in  a  preliminary  version  of  the 
functional  requirements  document  projected  as  many  as  100,000  users, 
based  on  past  experience  at  earth  science  data  centers.  However,  a 
preUminary  study,  prepared  in  1991  by  an  EOS  investigator,  estimated  a 
maximum  of  10,000  users  in  1992,  based  in  part  on  the  number  of 
members  in  the  mjyor  earth  science  professional  associations  and  users  of 
selected  data  centers.  This  estimate  does  not  include  nonresearch  users 
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such  as  those  at  state  and  federal  regulatory  agencies.  Although  NASA  now 
believes  that  the  EOSDIS  user  estimate  is  closer  to  10,000,  it  has  not  issued 
an  official  estimate.  Neither  the  10,000-  nor  the  100,000-user  estimate 
appears  in  the  final  functional  requirements  document  issued  to 
prospective  EOSDls  contractors.  However,  this  document  does  indicate  that 
the  EOSDIS  accounting  system  should  be  designed  to  track  information  on 
100,000  users. 

The  final  EOSDIS  functional  requirements  document,  which  is  part  of  the 
RFP,  also  includes  an  estimate  of  the  number  of  data  queries  EOSDIS  should 
be  able  to  process,  but  the  EOS  project’s  data  manager  noted  that  this 
figure  is  only  preliminary.  The  functional  requirements  indicate  that  NASA 
expects  an  average  of  400  user  queries  an  hour,  consisting  of  directory 
searches,  status  checks  on  data  requests,  or  queries  to  access  other  data 
base  functions.  Depending  on  their  nature,  these  queries  could  be  very 
resource-intensive,  making  heavy  demands  on  EOSDIS’  communications 
resources  and  data  handling  functions.  The  system  is  also  expected  to 
support  100  simultaneous  users.  According  to  EOS  project  officials,  these 
estimates  are  more  meaningful  in  designing  the  system  than  in  estimating 
the  number  of  users  the  system  is  expected  to  serve. 

To  determine  the  above  user  work  load  estimates,  NASA  conducted  a  study 
in  1990  that  involved  compiling  information  on  data  requests  and  queries 
from  several  data  centers.  The  study  found  considerable  uncertainty 
regarding  user  activity.  For  example,  it  notes  that  there  is  no  common 
understanding  of  current  data  center  activity  and  that  a  better  estimate  of 
the  size  of  the  user  community  and  how  often  users  will  interact  with 
EOSDIS  is  needed.  Although  the  study  did  identify  preliminary  work  load 
estimates,  it  T»,commended  further  review  and  modeling  of  the  daily  and 
annual  distribution  of  user  activity. 

Any  future  reviews  are  expected  to  be  conducted  by  the  EOSDIS  contractor. 
The  EOS  project  data  manager  said  that  the  contractor  is  expected  to 
provide  a  more  specific  estimate  of  the  number  of  simultaneous  users 
querying  the  system  and  a  better  assessment  of  the  actual  size  of  the 
EOSDIS  user  community.  These  reviews  will  then  be  used  in  designing  and 
sizing  the  system. 

As  more  definitive  information  is  obtained  on  the  user  community  and  its 
associated  work  load,  the  EOSDIS  design  must  be  flexible  enough  to 
incorporate  changes  in  system  processing  and  other  requirements. 
According  to  the  National  Academy  of  Sciences,  EOSDIS  should  have  a 
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system  architecture  sufficiently  flexible  to  accommodate  such  changes  and 
implement  them  in  an  evolutionary  manner."’  Furthermore,  the  Academy 
concluded  that  EOSDIS  poses  significant  continuing  challenges  and  must 
have  an  ongoing  mechanism  for  acquiring  advice  from  the  user 
community. 


Conclusions 


The  science  community,  internal  steering  groups  and  committees,  and 
others  have  urged  NASA  over  the  past  decade  to  seek  broad  user 
involvement  throughout  the  development  of  scientific  data  systems. 
Because  EOSDiS  is  intended  to  serve  the  research  needs  of  a  broad 
community  of  federal,  domestic,  and  international  users,  it  is  essential  that 
NASA  ensure  that  the  needs  of  these  users  are  incorporated  in  the  EOSDIS 
design.  However,  NASA  has  not  yet  obtained  broad  user  input  noi  has  it 
established  strong  mechanisms  for  obtaining  such  input  as  the  system 
evolves.  At  the  same  time,  there  is  considerable  uncertainty  about  the  total 
number  of  eventual  users  and  the  work  load  they  will  generate. 

Some  have  expressed  confidence  in  NASA’s  ability  to  design  EOSDIS  to  meet 
the  needs  of  the  broad  user  community  based  on  input  provided  almost 
exclusively  by  EOS  investigators.  We  disagree  with  this  belief.  In  fact,  we 
find  it  difficult  to  believe  that  a  successful  EOSDIS  system  development  wUl 
result  from  a  strategy  that  has  thus  far  failed  to  obtain  input  in  aetermining 
or  validating  requirements  from  most  of  the  federal  agencies  participating 
in  the  Global  Change  Research  Program,  as  well  as  thousands  of  other 
potential  users. 

The  process  by  which  user  needs  are  incorporated  over  time  is  also  a 
critical  element  in  the  future  success  of  EOSDIS.  Continuous  and  systematic 
input  from  the  broad  earth  science  research  community  is  needed  to 
ensure  that  appropriate  user  requirements  are  incorporated  into  the  design 
of  the  system. 


Global  Change  Research  Program:  An  Assessment  of  PTi)  1 Plans,  National  Academy  of  Sciences' 
National  Research  Council.  National  Academy  Press,  1990 
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Recommendation 

We  recommend  that  the  NASA  Administrator  strengthen  input  from  the  full 
range  of  anticipated  eosdis  users  by  either  establishing  a  new  independent 
committee  that  represents  the  broad  global  change  research  community  or 
strengthening  existing  advisory  groups,  such  as  the  eosdis  Advisory  Panel, 
to  achieve  the  necessary  broad  representation.  In  addition  to  the  EOS 
investigators,  the  committee  or  committees  should  represent  federal 
agencies  involved  in  global  change  research,  earth  science  researchers  not 
affiliated  with  NASA,  and  foreign  and  commercial  organizations  involved  in 
global  change  studies.  The  objectives  of  these  committees  should  include 

•  providing  for  the  dissemination  of  information  about  EOS  and  EOSDIS  to  the 
global  change  research  community; 

•  evaluating  the  EOSDIS  RFP  and  the  EOS  data  requirements  to  determine 
whether  they  meet  the  needs  of  the  user  community  and,  if  not, 
recommending  specific  modifications;  and 

•  monitoring  the  continued  evolution  of  EOSDIS  throughout  the  contractor’s 
development  of  the  system  to  ensure  that  EOSDIS  will  meet  the  broad  user 
community’s  needs. 

Agency  Comments 

As  requested,  we  did  not  provide  a  draft  of  this  report  to  NASA  for  its  review 
and  comments.  However,  we  discussed  the  report’s  contents  with  NASA 
officials,  including  the  Deputy  Director  for  Global  Change  and  the  Chief  of 
the  Modeling,  Data,  and  Information  Systems  Program  Office;  and  included 
their  conunents  as  appropriate.  The  NASA  officials  generally  agreed  with 
our  findings  and  recommendation.  Although  the  officials  believe  that  NASA 
has  been  seeking  substantial  consultation  with  other  federal  agencies  and 
international  partners,  they  agreed  that  it  would  be  beneficial  to  maintain 
an  awareness  of  how  the  international  community,  other  government 
agencies,  and  commercial  users  plan  to  use  EOSDIS. 

Our  audit  work  was  performed  in  accordance  with  generally  accepted 
government  auditing  standards  between  July  1991  and  March  1992.  As 
arranged  with  your  office,  unless  you  publicly  announce  the  contents  of 
this  report  earlier,  we  plan  no  further  distribution  until  30  days  from  the 
date  of  this  letter.  We  will  then  give  copies  to  other  appropriate 
congressional  committees,  the  Administrator  of  NASA,  and  other  interested 
parties  upon  request. 
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This  work  was  performed  under  the  direction  of  Samuel  W.  Bowlin, 
Director  for  Defense  and  Security  Information  Systems,  who  can  be 
reached  at  (202)  336-6240.  Other  nugor  contributors  are  listed  in 
appendix  II. 


Sincerely  yours. 


Ralph  V.  Carlone 
Assistant  Comptroller  General 
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Objectives,  Scope,  and  Methodology 


The  Conunittee  on  Science,  Space,  and  Technology  requested  that  we 
review  various  aspects  of  NASA’s  EOSDIS  program.  Based  on  this  request 
and  subsequent  discussions  with  Conunittee  representatives,  we  initiated 
separate  studies  on  NASA’s  EOSDIS  development  approach,  user  input  into 
EOSDIS  requirements,  and  eos  data  policy.  A  report  on  NASA’s  EOSDIS 
development  approach  was  issued  in  February  1992.‘  'This  report  covers 
the  second  topic  on  user  input  into  EOSDIS  requirements.  Specifically,  the 
Committee  requested  that  we  focus  on  the  extent  to  which  domestic  and 
international  users  (both  research  and  commercial)  have  provided  input 
into  the  definition  of  EOSDIS  requirements. 

To  obtain  background  on  user  involvement  in  past  data  systems  planning, 
we  reviewed  reports  by  the  National  Academy  of  Sciences’  National 
Research  Council,  including  reports  by  the  Committee  on  Data 
Management  and  Computation. 

To  determine  the  extent  of  user  involvement  in  EOSDIS,  we  reviewed  the 
NASA  process  for  obtaining  input  during  the  early  design  and  development 
phase  of  the  system  and  also  reviewed  NASA’s  future  plans  for  obtaining 
user  involvement.  To  do  this,  we  analyzed  NASA  Eos  documents  and 
interviewed  NASA  EOS  program  and  project  officials. 

Specifically,  we  reviewed  the  process  NASA  used  to  develop  the  three  main 
EOSDIS  requirement  and  planning  documents.  'These  documents  were 

•  the  Functional  and  Performance  Requirements  Specification  for  the  Earth 
Observing  System  Data  and  Information  System  Core  System,  July  1, 1991 
(EOSDIS  Functional  and  Performance  Requirements); 

•  the  Earth  Observing  System  Output  Data  Products  and  Input 
Requirements,  August  1991  (EOS  Instrument  Data  Requirements);  and  the 

•  Science  Data  Plan  for  the  EOS  Data  and  Information  System  Version  0  And 
Beyond,  (Draft)  September  1991  (EOSDIS  Data  Plan). 

To  assess  this  process,  as  well  as  to  identify  the  user  and  work  load 
estimates  and  the  role  of  EOSDIS  users,  we 

•  interviewed  the  eos  project  officials  responsible  for  each  requirement  and 
planning  document  and  members  of  the  EOSDIS  Advisory  Panel; 


'E;aith  Obaervlnj!  System;  NASA's  EOSDIS  Development  Approach  Is  Risky  (GAO/IMTEC-92-24,  Feb. 
25, 1992). 


Page  16 


6AO/llITEC>92>4e  The  EOSDIS  Uaer  Commniiltir 


Appendix  I 

Objectives,  Scope,  and  Methodology 


•  reviewed  documentation  to  identify  procedures  or  mechanisms  for 
obtaining  user  input  and  to  identify  the  user  and  work  load  estimates; 

•  obtained  information  on  dates  of  meetings  where  EOSDIS  requirements 
were  to  be  reviewed  by  the  user  community  and  listings  of  the  attendees; 
and 

•  obtained  listings  from  the  two  NASA  contractors  regarding  users  contacted 
during  their  functional  performance  requirement  studies. 

Further,  we  interviewed  scientists  in  the  earth  science  research  community 
regarding  user  input  into  EOSDiS.  The  scientists  we  interviewed  were 
selected  because  we  believe  they  represent  a  broad  range  of  potential 
EOSDIS  users,  including  NASA-affiliated,  nonaffiliated,  and  commercial 
researchers.  Among  the  scientists  were 

•  members  of  EOS  Engineering  Review  Committee; 

•  officials  of  the  Office  of  Science  and  Technology  Policy’s  Committee  on 
Earth  and  Environmental  Sciences  (GEES); 

•  members  of  the  GEES  Interagency  Working  Group  for  Data  Management 
and  Global  Change; 

•  representatives  of  the  six  key  earth  science  professional  associations  and  a 
mjgor  trade  association  of  industrial  users;  and 

•  EOS  investigators,  DAAG  representatives,  and  researchers  involved  in  NASA’s 
joint  research  efforts  with  other  federal  agencies. 

To  identify  NASA’s  plans  for  future  input,  we  interviewed  the  responsible 
EOS  officials,  obtained  a  listing  of  the  members  of  each  of  the  daag  user 
working  groups,  and  reviewed  internal  NASA  documents.  These  documents 
included  the 

•  EOS  Configuration  Management  Plan,  January  1990,  and 

•  EOS  Project  Configuration  Management  Procedures  Handbook,  July  1991. 

Our  audit  work  was  performed  in  accordance  with  generally  accepted 
government  auditing  standards  between  July  1991  and  March  1992  at 
various  locations,  including  NASA  headquarters  in  Washington,  D.C.;  the 
Goddard  Space  Flight  Center  in  Greenbelt,  Maryland;  and  the  Jet 
Propulsion  Laboratory  in  Pasadena,  California. 
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